INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINU  22202-2884 


REPORT 

NO.  92-002  October  9,  1991 

MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  DEFENSE  (FORCE  MANAGEMENT 

AND  PERSONNEL) 

SUBJECT:  Audit  Report  on  the  Operation  and  Modification  of 
Flight  Simulator  Training  Devices 
(Project  No.  OPC-0011) 


Introduction 

This  is  our  final  report  on  the  operation  and  modification 
of  flight  simulator  training  devices#  provided  for  your 
information  and  use.  The  objectives  of  the  audit  were  to 

determine  if  flight  simulator  training  devices  were  modified  in  a  , 
timely  and  effective  manner  that  satisfied  requirements  for  the 
training  of  aircrews.  We  also  evaluated  internal  controls 
applicable  to  the  modification  of  flight  simulator  training 
devices.  The  audit  was  performed  from  November  1989  to  December 
1990  and  was  initiated  because  of  a  previously  reported  condition 
that  tactical  flight  simulators  were  not  fully  compatible  with 
the  aircraft  they  simulated. 

Scope  of  Audit 

The  Military  Departments  operate  about  800  complex  flight 
simulators  worldwide.  We  could  not  determine  the  total  value  of 
these  simulators  because  of  their  ages  and  the  unknown  cost  of 
previous  modifications.  However#  the  63  flight  simulators 
included  in  our  audit  had  an  acquisition  cost  of  about 
$1.1  billion.  We  reviewed  in  detail  the  available  modification 
documents  for  these  flight  simulators  and  the  associated 
aircraft.  Enclosure  1  lists  the  activities  visited  or  contacted 
during  the  audit. 

This  program  audit  was  made  in  accordance  with  auditing 
standards  issued  by  the  Comptroller  General  of  the  United  States 
as  implemented  by  the  Inspector  General#  DoD#  and  accordingly 
included  such  tests  of  internal  controls  as  were  considered 
necessary. 

Internal  Controls 


This  audit  and  reviews  by  the  Services  showed  that  in  the 
past#  controls  were  not  in  place  to  ensure  that  flight  simulator 
training  devices  were  modified  concurrently  with  changes  in 
aircraft.  As  a  result#  training  of  aircrews  could  have  been 
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adversely  affected.  We  found  that  the  Services  recognize  this 
problem  and  continue  to  address  it. 

Background 

Plight  simulators  are  hardware  and  software  designed  to 
demonstrate  a  concept  or  simulate  an  operational  environment  for 
training  purposes.  They  instill  habits  and  decrease  reaction 
time  to  a  given  situation  and  imitate  the  functional  and 
physical  characteristics  of  operational  aircraft. 

Activities  periodically  evaluate  the  usefulness  of  their 
flight  simulators.  The  Services  use  the  results  of  these 
evaluations  to  improve  training  curriculums  and  ensure  that 
flight  simulators  meet  the  training  requirements  of  using 
commands . 

Flight  simulators  are  required  to  be  designed  and  developed 
as  part  of  operational  weapon  systems.  DoD  Directive  1430.13, 
"Training  Simulators  and  Devices,"  April  22,  1986,  requires 
concurrent  development  of  weapon  systems  and  training  systems. 
DoD  Directive  5000.53,  "Manpower,  Personnel,  Training,  and  Safety 
(MPTS)  in  the  Defense  System  Acquisition  Process,"  December  30, 
1988,  states  that  DoD  policy  is  to  enhance  the  operational 
suitability  and  effectiveness  of  initial  procurements  and 
modifications  of  all  systems,  including  simulators. 

Prior  Audit  Coverage 

The  Naval  Audit  Service  issued  Report  No.  021-S-89, 
"Multilocation  Audit  of  Requirements  Determination,  Utilization, 
and  Effectiveness  for  Training  Devices,"  January  18,  1989.  This 
audit  found  that  the  Chief  of  Naval  Operations  had  not 
established  goals  and  standards  for  the  use  of  flight  simulators 
and  that  utilization  data  had  been  improperly  collected  and 
reported.  Also,  improper  procedures  were  used  to  project  new 
requirements  for  aviation  simulators.  Use  of  improper  procedures 
resulted  in  the  procurement  of  incorrect  quantities.  In 
addition,  the  Chief  of  Naval  Operations  had  not  developed 
procedures  for  certifying  the  use  of  aviation  training  devices  as 
a  substitute  for  actual  flight  time.  The  Naval  Audit  Service 
considered  management's  responses  adequate. 

Discussion 


Our  audit  determined  that  flight  simulators  were  not 
modified  concurrently  with  changes  to  operational  aircraft.  This 
condition  occurred  because  modifications  to  simulators  and 
modifications  to  operational  aircraft  were  not  funded  early  or 
simultaneously.  We  also  found  that  the  Services  were  not 
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evaluating  the  effectiveness  of  training.  Therefore,  the 
Services  were  not  able  to  identify  deficiencies  in  simulators  and 
assess  the  overall  effectiveness  of  these  devices. 

Modifications  to  Flight  Simulators.  Flight  simulators 
were  not  being  modified  concurrently  with  aircraft  modifications 
because  needed  modifications  were  not  identified  promptly.  Also, 
funding  was  not  designated  or  available  early  in  the  modification 
process. 

Comprehensive  documentation  was  not  always  available  to 
determine  how  long  modifications  to  simulators  lagged  behind 
modifications  to  the  operational  aircraft  they  were  designed  to 
represent.  Ideally,  improvements  to  simulators  should  have  been 
developed  in  parallel  with  aircraft  to  ensure  that  new  training 
capabilities  were  available.  Specific  examples  of  outdated 
simulator  configurations  are  discussed  below; 

o  F-15E  Simulator  -  The  Air  Force  used  two  F-15E 
weapon  system  trainers.  There  were  significant  differences  in 
the  operational  flight  program  software  in  the  aircraft  and  the 
simulators.  This  affected  the  air-to-air  and  air-to-ground 
radars,  and  the  targeting  and  navigation  capabilities  of  the 
simulator . 


o  F-14A  Simulator  -  The  Navy  had  four  operational 
flight  trainers  and  two  weapon  system  trainers  to  support 
training  on  the  F-14  aircraft.  Because  updated  computer 
software  was  scheduled  to  be  installed  up  to  2  years  late, 
shortfalls  existed  in  simulations  of  electronic  warfare,  full 
weapon  systems  threat  modeling,  and  simulations  of  the  pod  system 
for  tactical  air  reconnaissance. 

o  SH-2  and  SH-60  Simulators  -  The  Navy's  inventory 
included  weapon  systems  and  operational  flight  trainers.  The 
simulators  were  deficient  in  acoustic  training  capabilities  and 
had  discrepancies  in  flight  fidelity  (the  ability  to  represent 
aircraft  performance,  characteristics,  and  environment)  and 
autorotation.  The  SH-2  weapon  system  trainers  had  such  poor 
visual  cues  that  they  were  used  only  as  procedural  trainers. 

Funding  for  Modifications  of  Flight  Simulators. 
Modifications  to  training  systems  and  aircraft  generally  had  not 
been  funded  together.  For  example,  the  Navy  funded  $4.6  million 
of  the  $14.9  million  needed  to  upgrade  the  SH-2  weapon  system 
trainer.  The  Air  Force's  unfunded  requirements  for  the  F-15E 
weapon  system  trainer  totaled  $143  million  for  engineering 
changes,  software  support,  and  visual  systems. 

DoD  Directive  5000.53  requires  the  Services  to  review  the 
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funding  for  training  simulators  and  address  the  impact  of  any 
shortfalls.  We  could  not  find  any  documentation  to  show  that  the 
Services  complied  with  this  requirement. 

Training  Evaluations.  Training  evaluations  are  used  to 
assess  the  effectiveness  of  flight  simulators  and  to  determine  if 
simulators  provide  useful  training  in  specific  tasks. 
Evaluations  are  a  method  of  documenting  deficiencies  in  the 
design  of  flight  simulators  when  these  deficiencies  limit  or 
prevent  effective  training.  The  Services  did  not  always  evaluate 
training  as  required  by  DoD  and  Service  guidance,  and  could  not 
uniformly  show  that  simulators  satisfied  the  training  objectives 
of  the  using  commands. 

We  found  examples  of  flight  simulator  configurations  that 
were  outdated  compared  to  actual  aircraft.  However,  we  could  not 
determine  how  this  affected  the  Services'  ability  to  train 
aircrews . 

Failure  to  perform  training  evaluations  decreases  the 
Services'  ability  to  assess  the  performance  of  flight 
simulators.  The  lack  of  training  evaluations,  simulator 
certifications,  and  quality  reviews  means  that  the  Services 
cannot  confirm  that  simulators  will  be  modified  in  the  future. 
This  deficiency  will  also  limit  the  Services'  ability  to 
recommend  modifications  to  simulators  in  the  future. 

To  compensate  for  improperly  configured  flight  simulators, 
the  Services  often  used  alternative  methods  to  train  pilots. 
Flight  hours  and  increased  student/instructor  interaction  were 
used  to  make  up  shortfalls  in  flight  simulators. 

Conclusion 


The  Services  have  documented  the  fact  that  flight  simulators 
and  aircraft  have  not  been  modified  concurrently.  An  internal 
Navy  report  acknowledged  deficiencies  in  this  area,  and 
recommended  parallel  funding  and  the  establishment  of  an 
effective  process  for  evaluating  simulators.  In  FY  1984,  a 
General  Officer  Simulator  Broad  Area  Review  by  the  Air  Force 
addressed  the  problem  of  simulators  that  were  not  modified 
concurrently  with  the  aircraft  they  supported.  The  review 
identified  several  problems:  conflicts  in  training  priorities, 
noncompliance  with  Air  Force  policy  for  flight  simulators,  and 
late  funding.  Recommendations  included  early  and  protected 
funding  for  simulators,  timely  direction  of  the  simulator 
program,  better  definition  of  requirements,  and  ensuring  that 
simulator  modifications  receive  the  same  priority  as  aircraft 
modifications.  In  addition,  an  Air  Force  Inspector  General 
Report  issued  in  December  1990  described  the  adverse  operational 
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impact  and  additional  costs  incurred  when  modifications  to  flight 
simulators  and  aircraft  are  not  funded  and  installed 
concurrently. 

Since  the  Services  have  recognized  and  continue  to  address 
the  major  problems  with  flight  simulators,  no  recommendations 
were  made  in  our  report. 

Management  Comments 

We  provided  a  draft  of  this  report  to  the  addressee  on 
June  28,  1991.  Because  there  were  no  recommendations,  no 
comments  were  required  from  management,  and  none  were  received. 
Therefore,  we  are  publishing  this  report  in  final  form.  Copies 
will  be  distributed  to  the  activities  listed  at  Enclosure  3. 

The  courtesies  extended  to  the  audit  staff  (listed  at 
Enclosure  2)  are  appreciated.  If  you  have  any  questions  about 
this  audit,  please  contact  Mr.  James  G.  McGuire  at  (804)  766-9108 
or  Mr.  Timothy  J.  Tonkovic  at  (804)  766-3319.  We  will  give  you  a 
formal  briefing  on  the  results  of  the  audit  within  15  days  of  the 
date  of  this  memorandum,  should  you  desire  it. 


Enclosure 


Robert  J'.  Lieberman 
Assistant  Inspector  General 
for  Auditing 


cc: 

Comptroller  of  the  Department  of  Defense 
Assistant  Secretary  of  the  Army  (Financial  Management) 
Assistant  Secretary  of  the  Navy  (Financial  Management) 
Assistant  Secretary  of  the  Air  Force  (Financial 
Management  and  Comptroller) 


ACTIVITIES  VISITED  OR  CONTACTED 


Office  of  the  Secretary  of  Defense 

Assistant  Secretary  of  Defense  (Force  Management  and  Personnel), 
Washington,  DC 

Department  of  the  Army 

Deputy  Chief  of  Staff  for  Operations  and  Plans,  Washington,  DC 
D.S.  Army  Materiel  Command,  Alexandria,  VA 
D.S.  Army  Training  and  Doctrine  Command,  Fort  Monroe,  VA 
U.S.  Army  Aviation  Center,  Fort  Rucker,  AL 

Project  Manager  for  Training  Devices,  U.S.  Army  Materiel  Command, 
Orlando,  PL 

U.S.  Army  Aviation  Systems  Command,  St.  Louis,  MO 
Headquarters  III  Corps,  Fort  Hood,  TX 

Department  of  the  Navy 

Assistant  Chief  of  Naval  Operations  (Air  Warfare),  Arlington,  VA 
Commander  in  Chief,  D.S.  Atlantic  Fleet,  Norfolk,  VA 
Naval  Air  Systems  Command,  Arlington,  VA 

Commander  Naval  Air  Force,  U.  S.  Atlantic  Fleet,  Norfolk,  VA 
Naval  Training  Systems  Center,  Orlando,  FL 

Naval  Training  Systems  Center,  Atlantic  Region,  Portsmouth,  VA 
Fighter  Wing  One,  Oceana  Naval  Air  Station,  Virginia  Beach,  VA 

Department  of  the  Air  Force 

Office  of  the  Deputy  Chief  of  Staff,  Plans  and  Operations, 
Washington,  DC 

Air  Force  Systems  Command,  Aeronautical  Systems  Division, 
Wright-Patterson  Air  Force  Base,  OH 
Tactical  Air  Command,  Langley  Air  Force  Base,  VA 
Ogden  Air  Logistics  Center,  Hill  Air  Force  Base,  DT 
Air  Force  Inspection  and  Safety  Center,  Norton  Air  Force  Base,  CA 
832nd  Air  Division,  Luke  Air  Force  Base,  AZ 

Tactical  Air  Warfare  Center  Detachment,  Luke  Air  Force  Base,  AZ 
Non-DoD  Federal  Organizations 

U.S.  General  Accounting  Office,  Washington,  DC 
Non-Government  Activities 

Information  Spectrum,  Inc.,  Falls  Church,  VA 

Advanced  Engineering  &  Planning  Corporation,  Inc.,  St.  Louis,  MO 


ENCLOSURE  1 


AUDIT  TEAM  MEMBERS 


Nancy  L.  Hendricks,  Director,  Financial  Management  Directorate 

James  G.  McGuire,  Program  Director 

Timothy  J.  Tonkovic,  Project  Manager 

Scott  Grady,  Team  Leader 

Richard  Hanley,  Auditor 

Suzanne  Hutcherson,  Auditor 

James  R.  Knight,  Auditor 

Cheryl  D.  Smith,  Auditor 

Eva  M.  Zahn,  Auditor 

Susanne  B.  Allen,  Editor 


ENCLOSURE  2 


REPORT  DISTRIBUTION 


Office  of  the  Secretary  of  Defense 

Assistant  Secretary  of  Defense  (Force  Management  and  Personnel) 
Comptroller  of  the  Department  of  Defense 

Department  of  the  Army 

Secretary  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management) 
Department  of  the  Navy 
Secretary  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management) 
Comptroller  of  the  Navy 

Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and 
Comptroller) 

Non-DoD  Federal  Organizations 

Office  of  Management  and  Budget 

U.S.  General  Accounting  Office, 

NSIAD  Technical  information  Center 

Congressional  Committees; 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 

Senate  Ranking  Minority  Member,  Committee  on  Armed  Services 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Ranking  Minority  Member,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 
House  Subcommittee  on  Legislation  and  National  Security, 
Committee  on  Government  Operations 


ENCLOSURE  3 


INTERNET  DOCUMENT  INFORMATION  FORM 


A .  Report  Title:  Audit  Report  on  the  Operation  and  Modification  of  Flight 
Simulator  Training  Devices 


B.  DATE  Report  Downloaded  From  the  Internet:  06/27/99 


C.  Report’s  Point  of  Contact:  (Name,  Organization,  Address,  Office 
Symbol,  &  Ph  #):  OAIG-AUD  (ATTN:  AFTS  Audit  Suggestions) 

Inspector  General,  Department  of  Defense 
400  Army  Navy  Drive  (Room  801 ) 

Arlington,  VA  22202-2884 


D.  Currently  Applicable  Classification  Level:  Unclassified 

E.  Distribution  Statement  A;  Approved  for  Public  Release 

F.  The  foregoing  information  was  compiled  and  provided  by: 

DTIC-OCA,  Initials: _ VM _ Preparation  Date  06/27/99 


The  foregoing  information  should  exactly  correspond  to  the  Title,  Report  Number,  and  the  Date  on 
the  accompanying  report  document.  If  there  are  mismatches,  or  other  questions,  contact  the 
above  OCA  Representative  for  resolution. 


